Dynamic NMFs with Temporal Regularization for Online Analysis of Streaming Text

Learning a dictionary of basis elements with the objective of building compact data representations is a problem of fundamental importance in statistics, machine learning and signal processing. In many settings, data points appear as a stream of high dimensional feature vectors. Streaming datasets present new twists to the dictionary learning problem. On one hand, dictionary elements need to be dynamically adapted to the statistics of incoming datapoints, and on the other hand, early detection of rising new trends is important in many applications. The analysis of social media streams (e.g., tweets, blog posts) is a prime example of such a setting where topics of social discussions need to be continuously tracked while new emerging themes need to be rapidly detected. We formalize such problems in terms of online learning of data-dependent, dynamic non-negative dictionaries with novel forms of temporal regularization. We describe a scalable optimization framework for our algorithms and report empirical results on simulated streams and real-world news stories.